PHOTOGRAMMETRIC MEASUREMENT SYSTEM AND METHOD 



BACKGROUND OF THE INVENTION 
Cross-Reference to Related Application 

This application claims priority from provisional application 60/221 ,702, filed July 31 1 
2000, entitled "Photogrammetric Measurement System and Method." 

Field of the Invention 

The present invention relates to systems and methods for remote measurement of 
geometric features on an object, and, more particularly, to such systems and methods that 
are photogrammetrically based. 

Description of Related Art 

Photogrammetry is a fast, accurate three-dimensional measurement method based 
on photographic principles. In a single-camera or sequential mode of photogrammetry, a 
single high-resolution digital camera takes multiple pictures of an object from different 
locations. These pictures are then automatically processed to yield three-dimensional 
coordinates of points on an object (Ganci and Handley, 1998). 

The sequential mode is not real time; so it can only measure static objects and 
targeted points. However, it is well suited to high-accuracy measurement of large, 
complicated objects, since virtually any number of pictures can be taken and processed. 
In addition, the photography is usually quick; so production downtime is low, and 
temperature effects are minimized. The use of the sequential method of photogrammetry 



on a typical measurement is illustrated in FIG. 1, wherein the user 90 is shown taking 
pictures of an object 91 from three locations 71,72,73 sequentially. 

Sequential photogrammetry has other attractive attributes, including high accuracy, 
portability, and the ability to perform measurements in unstable or vibrating environments 
5 (Brown, 1998). All these attributes combined with the present invention are believed to 
make sequential photogrammetry a compelling choice for numerous applications in the 
automotive industry. 

Three-dimensional inspection within an automotive environment has been known 
to be conducted within a coordinate measurement machine (CMM) room. The CMM room 
10 q typically has several stationary gantry-type CMMs dedicated to a certain class of 
%Q dimensional inspection. This class of inspection is most notably characterized by the need 

as 
i 

to bring the part to be measured to the CMM. Such a constraint instantly disqualifies 
^ traditional CMMs from inspecting the tooling fixtures that are located on the assembly line. 
Ji** These fixtures are in fact the most important items measured in the factory, as they control 
15 y the dimensions and fit-up of the subassemblies throughout the manufacturing process. 

o In-line production measurements are also impossible using CMM systems. 

il 

It would thus be desirable to have a portable system that can measure items in 
place for a number of applications. Although numerous portable, in-place measurement 
systems exist, they typically rely on touching the desired features with some type of probe 
20 or measuring adapter that is usually held in place by an operator. For many applications, 
access to the object is limited, making setup of the instrument difficult. Furthermore, 



2 




touching the desired features with the measuring device can be difficult, tedious, and prone 
to lead to measurement errors. 

For many automotive inspection tasks it is necessary quickly to collect and process 
feature data. One limiting factor in the use of photogrammetry for feature measurement 
5 in the past has been the need for point-of-interest targeting. For example, if a plane is 
desired to be measured, then at least three points (> 3 for redundancy) need to be applied 
to the surface that defines that plane. Similar requirements apply to other geometric 
features, such as circles and lines. In some instances, point-of-interest targeting cannot 
be used to define the feature. 
10 q An edge is one example of a feature that cannot be directly targeted. Typically, a 

sQ special target adapter is needed to define the edge 92, or the edge is determined indirectly 
l ,0 via the intersection of two planes 93,94 (FIG. 2). The example of an edge can be extended 

I : I 

^ to a corner via the addition of another plane and a line-line intersection to produce the 

q corner. Traditionally, target adapters have been used to bypass some of the problems 

15 y associated with measuring difficult features such as edges and corners. The time penalty 

Ijl 

p for these adapters comes at the processing stage, when they need to be identified and 
reduced to yield the desired feature. 

An alternative to stick-on targeting when using photogrammetry is the use of hand- 
held probes to touch points of interest (Ganci and Brown, 2000). Three exemplary probes 
20 95-97 (FIGS. 3A-3C) each have a standard tip 98, similar to that used on conventional 
CMMs. In addition, each probe 95-97 has five permanent targets 99. In use, two or more 
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cameras simultaneously view these targets 99 and calculate their xyz position. From the 
xyz position of the five targets 99, the probe tip 98 position can be calculated. 

Probes have been successfully used in automotive applications for many years. 
However, in some applications setting up the cameras for measuring the object is difficult. 
Furthermore, many features are difficult to probe accurately, especially in the tight spaces 
and difficult conditions encountered in many measurements. 

Sequential photogrammetry can often acquire data rapidly since photography is very 
quick. However, sometimes the need to target the desired features is difficult and time 
consuming. If, instead, a probing tool is used, there is typically a setup time for the 
instrument followed by laborious probing and analysis of each feature. The present 
invention is directed to reducing setup time and also to removing the requirement for 
probing features. 

SUMMARY OF THE INVENTION 

It is therefore an object of the present invention to provide a system and method for 
measuring geometric features of an object. 

It is an additional object to provide such a system and method particularly adapted 
for difficult-to-measure features. 

It is a further object to provide such a system and method that use a targeted 
adaptor. 

It is another object to provide such a system and method that incorporate digital 
photogrammetry. 



It is yet an additional object to provide such a system and method that are adapted 
for use in the automotive industry. 

These objects and others are attained by the present invention, an embodiment of 
which is a method for characterizing a geometric element of an object. The method 
comprises the step of positioning a calibrated target adjacent a calibration geometric 
element. The calibrated target comprises at least two differentially detectable features 
having a known geometric relationship to each other. 

Next a relationship of the calibrated target to the calibration geometric element is 
determined, and the calibrated target is moved adjacent a geometric feature of an object 
desired to be characterized. Then photogrammetry is applied to the calibrated target 
features and the desired geometric feature to spatially characterize the desired geometric 
feature. 

Another embodiment of the invention is a system for characterizing a geometric 
element of an object. The system comprises a movable calibrated target comprising at 
least two differentially detectable features having a known geometric relationship to each 
other. The system also comprises a photogrammetric analysis system for determining a 
relationship of the calibrated target to a calibration geometric element. The analysis 
system is also for spatially characterizing a geometric feature of an object desired to be 
characterized using the calibrated target features. 

The features that characterize the invention, both as to organization and method of 
operation, together with further objects and advantages thereof, will be better understood 
from the following description used in conjunction with the accompanying drawing. It is to 



be expressly understood that the drawing is for the purpose of illustration and description 
and is not intended as a definition of the limits of the invention. These and other objects 
attained, and advantages offered , by the present invention will become more fully apparent 
as the description that now follows is read in conjunction with the accompanying drawing. 

BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 (prior art) illustrates the process of sequential photogrammetry. 
FIG. 2 (prior art) shows the use of a plane-plane intersection to derive the location 
of an edge. 

FIGS. 3A-3C (prior art) illustrate three exemplary configurations of hand-held 
probes. 

FIGS. 4A-4C illustrate a plane target's top view (FIG. 4A), rear view (FIG. 4B), and 
side view (FIG. 4C). 

FIG. 5 illustrates an exemplary calibration setup. 

FIG. 6 illustrates a calibration file for the setup of FIG. 5. 

FIGS. 7A-7L illustrate fourtypes of feature targets (Table 1): plane (FIGS. 7A-7C); 
edge (FIGS. 7D-7F); corner (FIGS. 7G-7I); and circle (FIG. 7J-7L). 

FIGS. 8A-8I illustrate sample FT solutions: for measuring a plane (FIGS. 8A,8B), 
an edge (FIG. 8C), a corner (FIGS. 8D, alone, and 8E,8F, together), a complex corner 
(FIG. 8G), and a centerline (FIG. 8H). 

FIGS. 9A,9B illustrate two case studies (Table 3): FIG. 9A illustrates fixture 
measurement; FIG. 9B, an in-line car body inspection. 



FIG. 10 is a side perspective view of a sample clamping mechanism. 
FIG. 11 illustrates a camera station network for the system of FIG. 10. 
FIGS. 12A,12B show the measurement of two features on the system of FIG. 10. 
FIG. 13 illustrates a test unit for measuring features of an axle carrier on an 
automobile. 

FIG. 14 shows the camera station network for FIG. 13. 

FIG. 15A shows key measurement features of the system of FIG. 13. 

FIG. 15B shows geometric reductions for the features of FIG. 12A. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

A description of the preferred embodiments of the present invention will now be 
presented with reference to FIGS. 4A-15B. 

Feature targets (FTs) are identified with the use of a "code target," which comprises 
a unique, calibratable pattern of geometric shapes, such as, but not intended to be limited 
to, squares, and a substantially central dot. The important element in the shapes is that 
there be at least two different shapes, preferably a unitary shape comprising a first shape 
and a plurality of shapes comprising a second shape. 

Means are provided for automatically detecting the patterns and for identifying and 
measuring them by image processing techniques (Fraser 1997a,b). 

The addition of such a target to an adaptor makes it possible to identify that adaptor 
in a measurement. There are advantages to automating the use and reduction of these 



adaptors. Once an FT is identified, a calibration file associated therewith is used to 
determine the type and function of that particular target adaptor. 

A first embodiment of the FT 1 0 is for measuring a plane. This FT (FIGS. 4A-4C) 
comprises a planar base 1 1 , comprising, for example, a flat piece of aluminum. The top 
5 side 121 of the base 11 has a plurality of coded targets 12,12' thereon (FIG. 4A); the 
bottom side 122 of the base 11 has at least one magnet 13 embedded therein to hold the 
FT 10 in place. This positioning device is exemplary and is not intended to be limiting, and 
it will be understood by one of skill in the art that alternate affixing methods are able to be 
substituted therefor, such as an adhesive for affixing to nonmetallic portions of objects. 
10^ The relationship between the targets 12,12 f on the top side 121 of the base 11 and the 

[a plane represented by the base 11 is determined by a one-time calibration. 

lu 

x 4 The FT 10 is placed on a flat surface 89 such as granite (FIG. 5). Retroreflective 

131 

! a targets 14 are placed adjacent the FT 10 and are used to define a "local" plane 15. A 

Q 

^ photog ram metric measurement is then made to determine the location of the face targets 
15 IZ 12 relative to the local plane 15, which thus serves as a calibration geometric element for 
a planar surface. 

With the relationship between the targets 12,14 and the local plane 15 known, the 
coded target and its adaptor form a feature target 10 capable of directly measuring any 
plane to which it is attached. The relationship between the targets 12 and the plane 15 is 
20 shown in FIG. 6, as measured by a camera at station 24. 

In an alternate embodiment, if a right-angle bracket is used instead of a flat plate, 
the resulting FT 1 0 after calibration can measure a plane at right angles to the face targets. 

8 



Multiple FTs can be placed on a block, and the feature planes they measure intersected 
to form a line 92 as shown in FIG. 2. 

The face targets 12 can be in any orientation to the plane that needs to be defined. 
For example, it might be desired to have the FT at a 45° angle to the plane. In fact, the 
targets 12 can be in any relative orientation providing that they can be calibrated and are 
thus predetermined at the time of the desired characterization. 

At run time, a local transformation is performed for each of the FTs found. This 
transformation uses the calibrated and measured values of the face targets 12 to transfer 
the plane, line, or feature desired into the global coordinate system of the measured object. 
Photogrammetric software 51 is known in the art that can be resident on a processor 50 
for performing the requisite calculations, the processor 50 in electronic communication with 
the camera stations 24. 

Four exemplary types of FTs are given in Table 1 and FIGS. 7A-7K. Exemplary 
advantageous features of FTs are given in Table 2. In FIGS. 7A-7C planar FTs 
10a, 10b, 10c are used to define contact planes 123a,123b,123c, with the bases 
11a,11b,11c shown at 0°, 90°, and 45°, respectively. 

Edge definitions are provided (FIGS. 7D-7F) by using FTs 10d,10e,10f to define 
two contact planes 123d,123d';123e,123e';123f,123f each, and then deriving their 
intersection, which comprises the desired edge 124a,1 24b, 124c. FT 10d is a mirror image 
of FT 10e, both having angles of substantially 90° between the contact planes. FT 1 0f is 
a tilted view of a target having an angle between the planes less than 90°. 



Corner definitions are provided (FIGS. 7G-7I) with three contact planes 
123g,123g\123g" to define two edges 123h,123h\ from which in turn are derived the 
corresponding corner 123i. 

A center point 128 of a circle, or of a cylindrical object 125, for example, may be 
defined with the use of an FT 10j having two planes 123j,123j' at substantially right angles 
(FIGS. 7J.7K) or an FT 10k having three planes 123k,1 23k', 123k" at substantially right 
angles to each other (FIG. 7L). In both cases the cylindrical object 125 is surrounded by 
two of the planes, creating thereby two contact points 1 26,1 26';1 27,1 27' at tangent points. 
From the positions of these contact points 126,1 26'; 127,1 27", as determined 
photogrammetrically, the radius and thereby the center point 128 may be derived. 



Table 1. Exemplary Types of Feature Targets 



Type 


FIG. 


Function 


Plane 


FIGS. 7A-7C 


contact plane defined 


Edge 


FIG. 7D-7F 


two contact planes and their intersection are defined 


Corner 


FIG. 7G-7I 


three contact planes, two edges, and the corresponding corner are 
defined 


Circle 


FIG. 7J-7L 


the center point of a circle with known radius is defined 



Table 2. Feature Target Features 




Measurement with FTs is much faster than with other methods. The setup time for 
sequential photogrammetry is very low compared with other in-place measurement 
systems. Using FTs to measure features is faster and easier in many applications than 
probing the feature. Photography is usually very quick. Finally, analysis is completely 
automated so that it is both fast and error-free. 
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Higher 
accuracy 


The accuracy obtained is higher than normal probed data because the FT is measured 
from multiple locations, and many points are used to define the feature. 


Line of Sight 


FTs can be configured and used to eliminate many line-of-sight problems. 


Versatile 


The method is extremely versatile in both its use and application. FTs can be 
configured to meet many measuring requirements. 


Minimal 
downtime 


The fast overall measurement and analysis time means critical production downtime is 
minimized. 


Enhanced 
targeting 


FTs can measure features that cannot be directly targeted. 


Repeat 
measure- 
ment 


The ability to automate measurements through construction macros (Ganci and 
Handley, 1998) makes FTs ideal for repeat production measurements. 



As an example of a use of the system and method of the present invention, FTs can 
be used to measure NC blocks on automotive fixtures. A target is placed on each block 
or surface desired to be measured. The target is preferably selected depending upon the 
data desired and the viewing angles available to the block. For example, for blocks having 
nonperpendicular faces, it is customary to use a combination of FTs. For pin or circle 
measurements it is necessary to specify the correct radius at the calibration phase, which 
may be obtained, for example, from the design data. 

Sample FT solutions are given in FIGS. 8A-8H. As indicated above, a plane 40 
may be measured with one FT 10 positioned on the plane 40, so long as the FT 10 has 
been properly calibrated (FIGS. 8A,8B); an edge 41 may be measured with an FT 10g 
having a first face 410 of the base 11g at right angles to a second face 411 of the base 
11g (FIG. 8C); a corner 42 may be measured in one step with two FTs 10h (FIG. 8D) or 
in two steps with a single FT 10i (FIG. 8E and 8F); a complex comer 43 may be measured 
in one step with three FTs 10j (FIG. 8G); and a centerline 44 of a cylindrical object 45, or 
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a cylindrical portion of an object, may be measured with in one step with two FTs 10k (FIG. 
8H). 

Two exemplary test case studies are illustrated in Table 3 and FIGS. 9A and 9B. 
For these two studies the FT concept of the present invention was believed preferable to 
other known prior art measurements. In FIG. 9A, a plurality of FTs 10 are positioned 
around the components of the device 46, permitting complex feature analysis without a 
hand-held probe. In FIG. 9B, an axle carrier 47 also has a plurality of FTs 10 positioned 
thereon. 



Table 3. Overview of Two Case Studies 



Case study 


Object, 
FIG. 


Objectives 


Case 1 : Fixture 
measurement 


FIG. 9A 


1 . Use FTs to inspect the fixture 

2. Complete the inspection in less time than the current 
inspection system 


Case 2: In-line car body 
inspection 


FIG. 9B 


1 . Use FTs to inspect the underbody of a vehicle during 
assembly (in-line) 

2. Complete the measurement in less than the production 
time of 5 min 



In Case 1 (FIG. 9A), analysis of production tooling requires a fast and efficient 
measurement system. The driving factors are time and availability for analysis. Case 2 
(FIG. 9B) illustrates the need for immediate information for decision making during 
production. In both cases the need for uninterrupted production schedules is deemed 
extremely important. 



In the case of production tooling measurement, unavailability of tooling during after- 
shift hours due to routine maintenance needs, and constantly changing production 
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schedules due to high product demand, has forced inspection personnel into working 
during scheduled break times. This means finding a way to measure a tool during a 
normal 40-min lunch break with the goal of eventually completing the measurement during 
a 10-min coffee break. 

5 The ability to flexibly analyze product deviation in-line is the focus of the second 

case study. Increasingly, inspection is needed at the point of origin in order to efficiently 
determine the cause of the deviation and set corrective action. Typical problems facing 
dimensional inspection include flexibility and time. The system needs to be set up quickly 
and measure in unstable environments, such as constantly moving assembly lines. It also 

O 

10 \u needs to be able to complete the measurement between assembly processes. 
=T Case 1 . In this study a 1 .8-m-long panel-holding fixture (FIG. 9A) is desired to 

hi 

q be measured, to determine the location of features such as corners, edges, and planes on 

m 

g " the fixture. 

o 

Sj An FT 10 was placed on each of the desired features. Where possible, one was 

Sii 

■="57 

15 ^ used. For features without a suitable target, the necessary data were created using a 

P 

f"* combination of FTs. 

An exemplary sample clamping mechanism is illustrated in FIG. 10, wherein four 
FTs 20-23 are shown. The FT 20 at the back of the clamp 88 is used to define a contact 
plane. The three remaining FTs 21-23 define the hard corner of the clamping surface. 
20 The fixture measurement required a total of 40 FTs to measure all the desired features. 

After targeting a total of 60 photographs were taken of the fixture 88. The number 
of photographs taken depends on the complexity of the measurement and accuracy 
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requirements. The photography for the fixture 89 was completed in approximately 5 min. 
Camera station locations 24 for the measurement are shown in FIG. 11. 
Statistics from the measurement of the fixture 89 are as follows: 



No. of photos 60 

5 No. ofFTs 40 

No. of scales 2 

Scale agreement 0.01 mm 

Accuracy rms (mm) xyz x 0.010 

y 0.009 

10 z 0.008 



Some of the features measured on the fixture 89 are shown in FIGS. 12A and 12B, along 
with the corresponding FT analyses 87,87'. 
^ To obtain a better idea of the time savings of the FTs of the present invention, the 

15^ same measurement was completed using conventional stick-on targeting and a 

W 

*j multicamera probe system. The probes were needed to collect data on the features that 

Co 

s could not be targeted. A comparison of the two measurements is given in Table 4, wherein 

M it is clear that the FT system is much faster. The automated analysis of the FTs especially 

ty 

^ saves time while also eliminating measurement errors. 

20 



Table 4. Comparison of the Two Measurements 



Measurement type 


FT solution 


Stick-on and probes 


Number of targets 


40 FTs 


320 


Targeting 


5 min 


25 min 


Probing 




15 min 


Target removal 


3 min 


20 min 


Photography 


2 min 


2 min 



14 



# 




Processing 


2 min 


3 min 


Analysis 


3 min 


30 min 


Total time 


15 min 


95 min 



10 q 



15 
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Case 2. This case study is a production measurement, wherein FTs were used 
to measure a front axle carrier 86 of an automobile, here, a BMW Z3. This test was 
performed to examine whether in-line measurements could be used to identify cars with 
bushing angle problems. Rectifying these problems early in the production process would 
ultimately result in significant scrap value savings farther down the line. The test unit 30 
is shown in FIG. 13. 

The measurement was desired to be performed within a time period the car would 
be idle at a station. A total of 5 min was set aside for targeting, photography, and tear 
down. Processing the images to yield the desired data was not necessary during the 5-min 
time limit. 

Two operators performed the measurement. The targeting and tear down were 
completed in approximately 2 min. A total of 18 photographs were collected in less than 
2 min with the network 31 used is shown in FIG. 14. A summary of statistics from the 
measurement follows: 



No. of photos 
No. of FTs 
No. of scales 
Scale agreement 
Accuracy rms (mm) xyz 



18 

8 

8 

0.01 mm 

x 0.008 

y 0.007 

z 0.017 
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Once the processing is finished, the desired features are automatically generated. 
In this case six planes, eight circles, and four lines were created. The feature generation 
is shown in FIGS. 15A and 15B, wherein the key measurement features are shown in FIG. 
15A, with plane template 31 and two rod templates 32 attached to the axle carrier 30 and 
5 the corresponding geometric reductions 33 in FIG. 15B. 

It can be seen that all the objectives of the study were met, illustrating how FTs 
could be used to automate and significantly reduce the amount of time needed to complete 
a measurement. 
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It may be appreciated by one skilled in the art that additional embodiments may be 
contemplated, including alternate shapes and configurations of the feature target and 
30 camera positions. 
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In the foregoing description, certain terms have been used for brevity, clarity, and 
understanding, but no unnecessary limitations are to be implied therefrom beyond the 
requirements of the prior art, because such words are used for description purposes herein 
and are intended to be broadly construed. Moreover, the embodiments of the apparatus 
illustrated and described herein are by way of example, and the scope of the invention is 
not limited to the exact details of construction. 
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